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PREFACE 


CUSTOMER  SATISFACTION  REPORTING 
CONDUCTED  BY  THE  DEFENSE  TECHNICAL  INFORMATION  CENTER 

The  Defense  Technical  Information  Center  (DTIC)  has  surveyed  its  registered  customers  since 
FY  1999.  There  are  two  surveys  conducted  yearly: 

Customer  Satisfaction  Survey  A  complete  sampling  (census)  method  of  all  users  excluding  the 
following:  DTIC  employees,  Air  Force  Materiel  Command  (AFMC)  personnel,  and  users 
identified  as  Top  200.  Surveys  were  conducted  in  FY  1999  through  FY  2004. 

Top  200  Users  Roughly  200  users  defined  in  terms  of  dollar  value  of  billing,  number  of 
documents  ordered,  and  downloads  from  Private  STINET. 


Customer  Satisfaction  Survey  Results:  How  Do  We  Measure  Up? 


Customer  Satisfaction  survey  findings  indicated  overall  that  the  majority  of  the  2004  survey 
respondents  were  satisfied  with  DTIC  services.  While  the  result  was  generally  positive,  the 
score  was  down  one  point  from  77  percent  to  76  percent  since  the  2003  survey.  The  DTIC  rating 
is,  however,  above  the  established  Federal  Government  American  Customer  Satisfaction  Index 
(ACSI)  baseline/benchmark  score.  The  ACSI  survey  was  conducted  in  December  2003.  The 
results  are  illustrated  in  the  table  below. 


Customer  Service 
Elements 

FY  2002 

CS  Survey 

FY  2003 

CS  Survey 

FY  2004 

CS  Survey 

DTIC  Customer 
Satisfaction  Rating 

81% 

77% 

76% 

ACSI  Federal 
Government 
Benchmark* 

71.1% 

70.2% 

70.9% 

DTIC  Excels  by 

+9.9 

+6.8 

+5.1 

*  ACSI  is  the  official  service  quality  benchmark  for  the  Federal  Government 

Note:  Data  relevant  to  this  chart  is  detailed  in  Part  II  of  this  report. 
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2004  DTIC  Customer  Satisfaction  Data  Summary 
Part  I:  Introduction 


Purpose:  This  report  contains  the  findings  of  the  2004  Defense  Technical  Information  Center 
(DTIC)  Customer  Satisfaction  (CS)  survey  conducted  in  July  2004.  The  purpose  of  the  survey 
was  to  help  DTIC  (1)  gauge  the  level  of  satisfaction  among  its  general  users  and  (2)  identify 
possible  areas  for  improving  its  products  and  services.  This  report  describes  the  survey  process, 
presents  its  results,  and  recommends  actions  identified  by  survey  respondents.  Specifically,  this 
report  summarizes  the  results  of  the  2004  CS  survey.  It  also  compares  findings  from  the  2002 
and  2003  CS  user  data.  Collectively,  the  report  allows  us  to  work  toward  the  following 
objectives: 

■  To  improve  customer  retention 

■  To  determine  the  perceived  quality  of  products,  services,  and  customer  care 

■  To  indicate  trends  in  products,  services,  and  customer  care 

■  To  benchmark  our  customer  satisfaction  results  with  other  federal  government  agencies 

To  ensure  that  we  gained  knowledge  about  these  objectives  from  our  users,  the  2004  CS  survey 
was  specifically  designed  for  querying  our  registered  users  in  the  five  (5)  categories  listed  below: 

■  Customer  Service  Experiences 

■  Global  Customer  Service  Performance 

■  DTIC  Offerings 

■  User  Demographics 

■  Communication/Access  and  Information  Requirements 

Specifically,  users  were  asked  to  answer  questions  geared  toward  customer  service  factors: 
accessibility,  accuracy,  courtesy,  helpfulness,  knowledge,  professionalism,  responsiveness,  and 
speed  of  service.  Other  questions  targeted  aspects  of  DTIC's  products  (i.e.,  customer 
involvement,  quality,  and  access)  as  well  as  infonnation  distribution  (i.e.,  adequacy,  utility,  and 
promptness). 

Methodology:  Web-based  and  e-mail  surveys  were  the  primary  collection  methods  selected  for 
this  effort.  In  addition,  one-on-one  telephone  follow-up  calls  were  used  to  gather  contact 
information  in  order  to  increase  the  response  rate.  There  were  7901  eligible  general  users  for  this 
survey  effort.  Active  users  not  surveyed  included  DTIC  employees,  users  identified  as  Top  200, 
and  Air  Force  Materiel  Command  (AFMC)  personnel  (AFMC  personnel  were  not  surveyed 
based  on  an  understanding  with  DTIC  management  to  exclude  this  group).  A  complete  sampling 
(census)  method  was  chosen  based  upon  (1)  the  potential  to  increase  response  rates  and  (2) 
knowing  that  anomalies  can  occur  and  have  occurred  when  launching  the  DTIC  survey  contact 
user  list.  After  three  email  contact  attempts,  1317  users  responded  to  the  survey.  Statistically, 
the  survey  results  are  reliable.  The  overall  results  of  the  survey  have  a  precision  of  +1-1  percent 
at  the  95%  level  of  confidence.  The  overall  response  rate  was  17  percent,  a  decrease  of  ten 
percentage  points  in  comparison  to  last  year's  response  rate. 


Possible  reasons  83  percent  of  the  2003  CS  survey  population  did  not  respond  to  the  survey  are: 

•  User  could  not  be  contacted  due  to  invalid  contact  infonnation  or  user  no  longer  with 
organization 

•  User  would  not  answer  survey  (too  busy,  does  not  complete  surveys,  not  familiar  enough 
with  DTIC  to  answer) 

•  Staff  shortages  in  the  Proactive  Customer  Advocate  Program  (PROCAP)  limited  the  calling 
effort  to  increase  response  rate 


Interpretation  of  the  Data: 

When  considering  the  results  of  Part  I  of  the  survey,  it  is  important  to  recognize  not  only  the 
characteristics  of  the  respondents,  but  also  the  characteristics  of  customers  who  did  not  receive 
the  survey  and  those  who  received  it  but  chose  not  to  respond.  Analysis  of  the  survey  results 
will  always  need  to  be  tempered  by  considerations  of  who  did  or  did  not  respond,  and  to  what 
extent  those  who  responded  have  the  same  characteristics  or  views  as  those  who  did  not  respond. 

Overall  favorable  percentages  are  based  on  a  normalized  data  process  which  uses  the  total 
number  of  valid  responses  along  a  five  point  scale  and  converts  these  to  a  100  point  scale.  These 
may  vary  somewhat  from  the  total  percentage  of  favorable  responses.  The  overall  favorable 
percentages  are  derived  from  the  “Mean  (normalized)”  column  in  the  tables  for  quantitative 
results  in  Appendix  B.  Most  analytic  conclusions  throughout  this  report  are  based  upon  these 
"overall  percentages." 
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Overall  Findings 


(Data  relevant  to  the  overall  findings  below  are  detailed  in  Appendix  B) 


Customer  Service  Issues 

■  Seventy-six  percent  of  surveyed  respondents  reported  that  they  were  satisfied  with  the  level 
of  DTIC's  customer  care.  (Reference  page  36) 

General  Findings 

■  Eighty-eight  percent  of  all  survey  respondents  indicated  that  DTIC  is  important  to  the 
accomplishment  of  their  business  objectives.  (Reference  page  45) 

■  Seventy-nine  percent  of  respondents  reported  that  DTIC  has  a  “Very  High”  to  "Medium" 
impact  in  supporting  their  overall  mission.  (Reference  page  46) 

■  Eighty-nine  percent  of  respondents  would  recommend  DTIC  to  colleagues. 

(Reference  page  46) 

User  Demographics 

■  The  majority  of  respondents  were  DoD  employees.  (Reference  page  41) 

■  The  majority  of  users  responding  came  from  three  user  groups:  Engineer, 

Researcher/ Analyst,  and  Librarian.  (Reference  page  41) 

■  This  year  the  CS  survey  results  showed  the  largest  number  of  respondents  classified  their  job 
position  as  Engineer  (21  percent).  Following  at  20  percent  was  the  Researcher/ Analyst  and 
Librarian  at  18  percent.  Although  these  three  job  positions  are  close  in  percentages,  this  is 
the  first  year  in  which  Librarians  weren’t  the  largest  number  of  respondents,  nor  in  fact  the 
second  largest.  (Reference  page  41) 

■  The  majority  of  the  respondents  (59  percent)  reported  they  obtain  infonnation  for  their  own 
use.  This  is  a  break  from  the  previous  trend  of  respondents  acting  primarily  as  information 
providers  to  others.  (Reference  page  42) 

■  The  largest  number  of  respondents  reported  being  registered  with  DTIC  for  less  than  one 
year.  (Reference  page  42) 
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DTIC  Online  Services 


■  Even  among  registered  users,  survey  data  showed  Public  STINET  as  the  most  widely  used 
DTIC  Online  Service  at  76  percent  in  comparison  to  Private  STINET  at  72  percent. 
(Reference  page  3  7) 

■  General  users  rated  the  Public  and  Private  STINET  Online  Services  quality  equally  at  sixty- 
six  percent.  (Reference pages  36  &  37) 

■  The  DTIC  Review  was  the  most  widely  used  product  and  service  excluding  Public  and 
Private  STINET.  (Reference  page  39) 

■  Ninety-two  percent  of  respondents  were  generally  satisfied  with  the  Registration  process, 
including  68  percent  who  rated  it  as  “Excellent”  or  “Very  Good.”  (Reference  page  40) 

Communications,  Access  and  Information  Requirements 

■  Close  to  half  of  DTIC’ s  general  population  who  responded  indicated  the  need  for  classified 
information.  (Reference page  47) 

■  Seventy-five  percent  of  respondents  indicated  that  they  are  able  to  find  the  type  of 
information  they  seek  from  DTIC.  (Reference  page  43) 

■  CS  survey  respondents  selected  Electronic  Mail  as  the  desired  means  to  be  informed  about 
DTIC  products,  services,  updates,  etc.  (Reference page  43) 

■  Surveyed  users  reported  they  are  satisfied  that  DTIC  does  an  adequate  job  communicating 
information  about  new  changes  and/or  enhancements  to  products  and  services  offerings, 
training  opportunities,  etc.  (Reference  page  44) 

■  Fifty-four  percent  of  2004  CS  respondents  surveyed  preferred  to  speak  to  a  "live  person" 
when  contacting  DTIC.  ( Reference  page  44) 
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Part  II:  Global  Customer  Service 


Performance  Rating  Review 


Overall  Customer  Satisfaction  Performance  Ratings: 

The  2004  CS  survey  findings  showed  that  76  percent  of  respondents  felt  satisfied  with  the  level 
of  DTIC's  customer  care.  When  this  rating  is  compared  to  2003  CS  survey  findings,  the  ratings 
differed  by  one  percentage  point.  The  chart  below  illustrates  the  percentage  of  customer  service 
satisfaction  over  a  three-year  period: 

Comparative  Analysis  of  Customer 
Satisfaction  (Customer  Care)  Performance  Rating 


Fig  2.1 


Asking  our  users  to  rate  their  level  of  satisfaction  with  DTIC’s  customer  care  quality  factors 
generates  the  customer  satisfaction  performance  rating.  The  eight  critical  care  elements  and 
response  data  are  displayed  in  Fig  2.2.  With  regard  to  aspects  of  this  year's  customer  satisfaction 
score,  a  majority  of  the  customer  care  factors  declined  in  their  rating.  Scores  for  Courtesy, 
Professionalism,  Helpfulness  and  Knowledge  continue  to  show  the  highest  score.  Although 
2004  scores  were  positive,  these  customer  support  staff  qualities  all  showed  a  decline  in  their 
scores  in  comparison  to  the  previously  surveyed  years.  Scores  for  Responsiveness,  Accuracy, 
Speed  of  Service,  and  Accessibility  continue  to  be  the  lowest  scoring  quality  factors.  It  should 
also  be  noted  that  these  factors  continue  to  remain  fairly  steady  in  comparison  to  the  previously 
surveyed  years. 
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DTIC's  Overall  Customer  Care 
Quality  Factors/Elements 

FY  2002 

CSS  Survey 

FY  2003 

CSS  Survey 

FY  2004 

CSS  Survey 

Performance 

Gap 

FY  2003-FY2004 

Accessibility  (Easy  to  access  staff) 

77% 

72% 

72% 

0 

Accuracy  (Provided  correct  info) 

80% 

76% 

75% 

-1 

Courtesy 

85% 

82% 

79% 

-3 

Helpfulness 

83% 

79% 

77% 

-2 

Knowledge 

83% 

78% 

77% 

-1 

Professionalism 

85% 

81% 

78% 

-3 

Responsiveness(Timely  response) 

78% 

75% 

75% 

0 

Speed  of  Service 

77% 

73% 

74% 

+1 

Overall  Rating 

81% 

77% 

76% 

Fig  2.2 


Comparative  Benchmarking  -  Comparison  to  American  Customer  Satisfaction  Index 

(AC  SI) 


DTIC's  Customer  Service  Results: 

■  DTIC’s  customer  service  is  compared  to  the  2003  Federal  Government  American 
Customer  Satisfaction  Index  (ACSI)  which  surveys  64  Federal  government  user  segments, 
including  35  Federal  government  Web  sites. 

■  DTIC's  composite  data  was  matched  for  comparability  with  the  common  ACSI  index  scoring 
factors. 

■  The  2003  aggregate  ACSI  score  for  the  Federal  government  rose  by  almost  one  (1)  percent 
from  70.2  to  70.9. 

■  DTIC’s  2004  Overall  Customer  Satisfaction  Survey  composite  score  is  76  percent,  a  decline 
of  one  percent  from  2003. 

*  Nevertheless,  when  measured  in  terms  of  the  latest  ACSI,  the  DTIC  rating  is  above  the 
established  federal  baseline/benchmark  score  by  5.1  points. 


Customer  Service 
Elements 

FY  2002 

CSS  Survey 

FY  2003 

CSS  Survey 

FY  2004 

CSS  Survey 

DTIC  Customer 

Satisfaction  Rating 

81% 

77% 

76% 

ACSI  Federal  Government 
Benchmark* 

71.1% 

70.2% 

70.9% 

DTIC  Excels  by 

+9.9 

+6.8 

+5.1 

*  ACSI  is  the  official  service  quality  benchmark  for  the  Federal  Government. 

Fig  2.3 
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Importance  of  DTIC  to  Accomplishment  of  Business  Objectives  and  Mission  Support: 


The  response  from  the  general  users  on  the  importance  of  DTIC  to  the  accomplishment  of  their 
business  objectives  continues  to  rate  high.  Eighty-eight  percent  of  all  survey  respondents 
indicated  that  DTIC  is  offering  a  valued  service  to  their  business  objectives.  In  addition,  close  to 
half  of  the  CSS  respondents  see  DTIC's  role  as  "very  to  extremely  important”  in  the 
accomplishment  of  their  business  objectives.  See  Fig  2.4  for  more  details. 
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FY  2002  CSS 
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FY  2004  CSS 

Survey  % 

Survey  % 

Survey  % 

□  Very  to  Extremely  Important 

52 

48 

49 

□  Important 

34 

37 

39 

□  Not  Very  Important  and/or  Not 

13 

15 

12 

At  All  Important 

Fig  2.4 


Seventy-nine  percent  of  CS  survey  respondents  reported  that  DTIC  has  a  “very  high”  to 
"medium"  impact  in  supporting  their  overall  mission  as  compared  to  seventy-six  percent  in  2003. 
See  Fig  2.5  for  more  details. 


Fig  2.5 
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Percentage 


Recommending  DTIC  Product(s)  and/or  Service(s)  to  Colleagues: 


Eighty-nine  percent  of  CSS  respondents  would  recommend  DTIC  to  colleagues.  This  rating  has 
continued  to  remain  fairly  steady  as  depicted  in  Fig  2.6. 
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Comparative  Analysis  on 
Recommending  DTIC  to  Colleagues 
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□  Yes 

91 

89 

89 

□  No 
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8 

10 

9 

Fig  2.6 
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Delivery  Processes: 

Forty-three  percent  of  respondents  reported  that  DTIC  meets  their  anticipated  delivery  schedules 
"every  time"  to  "most  of  the  time”  when  ordering  hard  copy  documents  in  contrast  to  7 1  percent 
of  respondents  in  FY  2002.  To  account  for  the  disparity  from  FY  2002  to  FY  2004  (this  data 
was  not  collected  in  FY  2003)  the  survey  questionnaire  must  be  referenced.  The  FY  2004 
questionnaire  incorporated  a  new  option:  “Rarely,  if  ever,  order  hard  copy.”  Fifty  percent  of  the 
respondents  for  the  FY  2004  survey  selected  this  option.  See  Figs  2.7  &  2.8  for  specifics.  It  is 
possible  that  many  of  this  fifty  percent  would  have  answered  either  “every  time"  to  "most  of  the 
time”  bringing  the  rating  more  in  line  with  FY  2002.  Nevertheless,  DTIC’s  internal  processes 
for  producing  and  delivering  hard  copy  documents  must  be  reviewed. 
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Customer  Satisfaction  with 
Meeting  Delivery  Schedules 


FY  2002  CS  Survey  %  FY  2003  CS  Survey%  FY  2004  CS  Survey% 

%  of  Satisfied  Users 


71 


N/A* 


43 


*Data  not  gathered 


Fig  2.7 


Customer  Satisfaction  with 

FY  2002 

CS  Survey 

FY  2003 

CS  Survey 

FY  2004 

CS  Survey 

Meeting  Delivery  Schedules 

Every  time  to  Most  of  the  time 

70% 

N/A* 

43% 

About  Half  the  time 

5% 

N/A* 

2% 

Never  to  Less  than  half 

2% 

N/A* 

5% 

Rarely,  if  ever,  order  hard  copy 

N/A* 

N/A* 

50% 

Download  Documents  themselves 

23% 

N/A* 

N/A* 

*Data  not  gathered 


Fig  2.8 
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Part  III:  DTIC’s  Offerings 

DTIC  Online  Services 


DTIC  Online  Service  Usage  Profile: 

Users  were  surveyed  on  their  usage  of  DTIC's  two  major  online  services— Public  STINET  and 
Private  STINET.  The  results  of  the  2004  CS  surveyed  findings  show  Public  STINET  is  the  more 
widely  used  of  the  two  at  76  percent,  followed  closely  by  Private  STINET  at  72  percent. 

See  Fig  3.1.  All  surveyed  users  are  eligible  to  use  Private  STINET,  which  has  a  wider  array  of 
resources  than  Public  STINET.  Reasons  for  the  heavier  usage  of  Public  STINET  will  be 
examined  in  depth  if  similar  results  are  obtained  in  2005. 

Online  Services  Usage 


Service 

FY  2002 

CS  Survey  Usage 

FY  2003 

CS  Survey  Usage 

FY  2004 

CS  Survey  Usage 

Public  STINET 

77% 

10% 

76% 

Private  STINET 

* 

45% 

72% 

Secure  STINET 

54% 

4% 

WED  (formerly 
Unclassified  DROLS) 

59% 

6% 

Classified  DROLS 

15% 

6% 

N/A**  | 

*  Private  STINET  introduced  April  2003,  replacing  Secure  STINET  and  WED 
**Data  not  captured 

Note:  For  FY  2003  -  Respondents  were  asked  to  write  in  an  answer  to  a  question  rather  than  select 
an  option.  An  additional  30%  of  respondents  wrote  in  "STINET"  as  an  answer  to  the  question 
without  specifying  Public,  Private,  or  Secure.  These  are  not  included  in  the  FY  2003  usage  profile 
chart 

Fig  3.1 


DTIC  Online  Services  Overall  Satisfaction  and  Performance: 

The  2004  CS  survey  respondents  were  asked  to  rate  their  level  of  satisfaction  with  two  of 
DTIC’s  online  services  -  Public  STINET  and  Private  STINET.  About  two-thirds  rated  the 
quality  of  each  as  “Very  Good”  to  “Excellent.”  See  Fig  3.2,  3.3. 


Accessibility,  Content,  and  Usability  were  the  top  three  rated  factors  while  Customer  Support 
was  the  lowest.  See  figures  3.2  and  3.3  depict  the  overall  rating  of  select  quality  factors  for  each 
STINET  service. 
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DTIC  Online  Services  Overall  Satisfaction  and  Performance: 


Accessibility 
Ease  of  Use 
Content 
Organization 
Usability 
Customer  Support 
Overall  Average 


Accessibility 
Ease  of  Use 
Content 
Organization 
Usability 
Customer  Support 
Overall  Average 


Public  STINET  Quality  Performance  Rating 


0  10  20  30  40  50  60  70  80  90  100 

(%) 


Fig  3.2 


Private  STINET  Quality  Performance  Rating 


10  20  30  40  50  60  70  80  90  100 

(%) 


Fig  3.3 
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DTIC's  Other  Products  and  Services  Usage  Profile: 

Excluding  Public  and  Private  STINET,  the  most  widely  used  product  or  service  that  was  offered 
by  DTIC  was  the  DTIC  Review  followed  by  ECAB  and  TR  Database  CD-ROM/DVD.  These 
results  are  slightly  lower  than  FY  2002  with  the  TR  Database  CD-ROM/DVD  showing  the 
greatest  decline  in  usage. 


Products  and  Services  Usage  Profile 


Service/Usage 

FY  2002 

CS  Survey 

FY  2003 

CS  Survey 

FY  2004 

CS  Survey 

ADD 

13% 

N/A* 

10% 

CAB 

19% 

N/A* 

10% 

ECAB 

25% 

N/A* 

20% 

Research  Summaries 

64% 

N/A* 

N/A* 

Automatic  Magnetic 

Tape  Dist. 

2% 

N/A* 

N/A* 

TR  Database  CD- 
ROM/DVD 

30% 

N/A* 

15% 

Retrospective 
Bibliography  Magnetic 
Tape 

1% 

N/A* 

N/A* 

The  DTIC  Review 

41% 

N/A* 

30% 

TRAIL 

19% 

N/A* 

12% 

*  Data  not  gathered 

Fig  3.4 
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DTIC's  Other  Products  and  Services  Quality  Rating: 


For  the  2004  CS  survey,  users  were  asked  to  rate  the  quality  of  each  product  and  service  shown 
in  Fig  3.5.  Ratings  were  largely  favorable.  Individually,  the  products  and  services  receiving  the 
highest  marks  for  overall  quality  satisfaction  were  ADD  and  ECAB  at  76  percent  each.  CAB 
was  the  lowest  rated  product  at  70  percent. 


User  Satisfaction  with  Individual  Products  and  Services 


FY  2004  DTIC's  Other  Products  and 

Favorable 

Neutral 

Unfavorable 

Services  Quality  Rating 

ADD 

76% 

18% 

5% 

CAB 

70% 

27% 

3% 

ECAB 

76% 

18% 

6% 

TR  Database  CD  ROM 

72% 

23% 

5% 

The  DTIC  Review 

71% 

24% 

5% 

Trail 

71% 

20 

9% 

Note:  “Not  Sure”  response  was  excluded  from  the  Chart. 


Fig  3.5 
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DTIC  Registration  Process 


DTIC  Registration  Process: 

The  Customer  Satisfaction  Survey  posed  two  registration  questions.  The  first  asked  the 
respondents  to  rate  their  overall  level  of  satisfaction  with  the  DTIC  registration  process.  The 
rating  was  68  percent  (. Reference  chart  on  page  47).  The  second  question  asked  the  respondents 
to  rate  their  level  of  satisfaction  on  four  quality  factors  -  Accessibility,  Ease  of  Use,  Speed  of 
Service,  and  Customer  Support.  The  overall  rating  was  7 1  percent  as  shown  in  Fig  3.6.  User 
comments,  immediately  following  the  figure,  may  provide  some  insight  as  to  the  level  of  rating. 


DTIC  Registration  Satisfaction  Rating 


FY  2004  Registration  Quality 
Factors 

Excellent 

Very  Good 

Good 

Fair 

Poor 

Not  Sure 

Accessibility  (Easy  to  access  site) 

23% 

40% 

25% 

5% 

3% 

4% 

Ease  of  Use  (Navigating  the  site) 

20% 

39% 

25% 

7% 

4% 

5% 

Speed  of  Service 

25% 

38% 

24% 

6% 

3% 

5% 

Customer  Support 

28% 

35% 

18% 

5% 

3% 

11% 

Overall  Satisfaction  Level:  71% 

Note:  Comparable  data  are  not  available  for  previously  surveyed  years 

Fig  3.6 


The  following  comments  represent  a  selected  sample  of  quotes  from  users  regarding  the 
registration  process: 

Pro: 

■  "Personally,  I’ve  never  had  any  troubles  with  DTIC  registration  or  at  least  anything  that  was 
not  minor.  My  recent  Private  STINET  password  change  went  fine  too." 

■  “Never  had  a  problem  -  seemed  quite  straight  forward.” 

■  "Online  steps  were  clear.  When  I  had  to  phone  DTIC  the  staff  were  a  great  help  and  very 
pleasant." 
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Con: 


■  "It  is  much  more  complicated  than  it  needs  to  be  and  takes  a  great  deal  longer  than  it  should 
particularly  for  those  of  us  who  are  re-applying  and  have  been  using  DTIC  Private  STINET 
since  its  inception  and  Public  STINET  before  that." 

■  "From  the  beginning  to  end  it  took  more  than  one  month.  There  were  problems  because  of 
misinterpretation  of  security  guidelines  dealing  with  a  contractor  working  on  government 
property." 

■  “Not  user  friendly  -  don’t  use  because  I  don’t  have  time  to  sort  through  the  awkward 

menus.” 

■  “I  registered  three  times  before  I  got  anything  other  than  the  email  automated  response  that 
my  registration  request  had  been  received.  The  process  took  weeks.” 

■  "The  registration  process  is  cumbersome.  Perhaps  it  is  necessary  in  the  light  of  security  but  it 
seems  to  me  that  it  could  be  streamlined." 


Requirement  for  Classified  Information 

Close  to  half  of  DTIC's  general  population  who  responded  indicated  the  need  for  classified 
information.  (Reference  chart  on  page  47) 


Requirement  for  Classified  Information 


Fig  3.7 
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Part  IV:  User  Demographics 


Organization  Type: 

The  customer  profile  in  this  survey  represents  five  (5)  organizational  groups  and  seven  (7)  job 
positions,  the  distribution  of  which  is  shown  in  Figures  4. 1  and  4.2.  Aspects  of  the  CS  survey 
findings  during  2002  through  2004  continue  to  show  similar  organizational  percentages, 
indicating  that  the  majority  of  respondents  (64  percent)  were  affiliated  with  the  Department  of 
Defense  (DoD).  Following  at  27  percent  was  the  Industry/Gov’t  Contractor.  Overall,  the 
respondents'  top  three  (3)  organizational  types  remain  consistent  with  previously  surveyed  years. 

User  Organization  Type 


Organization 

FY  2002  CS 
Survey 

FY  2003  CS 
Survey 

FY  2004  CS 
Survey 

DoD-Military/Civilian 
Services  Agencies 

48% 

53% 

64% 

Industry  /Gov't. 
Contractor 

38% 

33% 

27% 

Non-DoD  Federal  Gov't. 

5% 

4% 

4% 

Academia:  College/Univ. 

3% 

5% 

2% 

HBCU/MI 

3% 

2% 

1% 

Other 

N/A* 

3% 

2% 

*Data  not  gathered  in  previous  survey 


Fig  4.1 


Job  Position: 

The  job  positions  of  the  greatest  number  of  respondents  continued  to  reflect  the  same  three  (3) 
classifications  throughout  the  surveyed  years.  However,  in  2004  there  was  an  acknowledged 
change  in  the  order.  This  year  the  CS  survey  results  showed  the  largest  number  of  respondents 
classified  their  job  position  as  Engineer  (21  percent).  Following  at  20  percent  was  the 
Researcher/ Analyst  and  Librarian  at  18  percent.  Although  these  three  job  positions  are  close  in 
percentages,  this  is  the  first  year  in  which  Librarians  weren't  the  largest  number  of  respondents, 
nor  in  fact  the  second  largest. 
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Job  Position 


Job  Position 

FY  2002  CS 
Survey 

FY  2003  CS 
Survey 

FY  2004  CS 
Survey 

Librarian 

25% 

24% 

18% 

Researcher/Analysts/Program 

Manager 

16% 

16% 

20% 

Engineer 

17% 

17% 

21% 

Scientist 

9% 

11% 

9% 

Technical  Info  Specialist 

9% 

10% 

10% 

Contracting  Officer 

5% 

3% 

3% 

Resource 

Management/Financial/ 

Budget  Analyst 

4% 

3% 

2% 

Security 

N/A* 

N/A* 

N/A* 

Other 

15% 

17% 

17% 

‘Data  not  collected 

Note:  Grayed  entries  reflect  the  top  three  positions  in  terms  of  the  number  of  survey  respondents. 


Fig  4.2 

Intermediary  vs.  End  Users: 

The  two  categories  listed  in  Fig  4.3  note  how  our  respondents  acquire  scientific  and  technical 
information.  The  majority  of  the  respondents  (59  percent)  reported  they  obtain  information  for 
their  own  use.  This  is  a  break  from  the  previous  trend  of  respondents  acting  primarily  as 
information  providers  to  others. 


80- 

S>  60- 

re 

■*-< 

§  40- 

i- 

0) 

CL 

Comparative  Analysis  on 
Intermediary  vs.  End  User 

^U- 

u- 

FY  2002  % 

FY  2003  % 

FY  2004  % 

■  Intermediary  Users 

65 

55 

41 

□  End  Users 

35 

45 

59 

□  Other 

N/A* 

N/A* 

N/A* 

‘Data  not  gathered  in  previous  survey  Fig  4.3 
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Time  as  a  Registered  User: 

The  results  from  this  year's  survey  showed  that  the  largest  number  of  respondents  have  been 
registered  for  "Less  than  1  year."  The  total  number  of  active  registrants  rose  15.6%  from  July 
2003  to  July  2004.  Sixty-nine  percent  reported  having  been  registered  with  DTIC  two  (2)  years 
or  less,  while  3 1  percent  reported  three  (3)  years  or  more.  Also  to  be  noted  are  the  results  of  the 
cross  tab  data  indicating  the  Librarian  respondents  represent  the  largest  number  of  “6  or  more 
years”  registered  users.  The  Engineers  represented  the  majority  of  “1  to  2  years”  as  registered 
users.  The  data  displayed  in  Fig  4.4  define  respondents’  time  as  a  registered  DTIC  user. 
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Comparative  Analysis  on 
Time  Data 


u- 

FY  2003  % 

FY  2004  % 

□  Less  than  1  year 

10 

41 

□  1-2  years 

41 

28 

□  3-5  years 

24 

16 

□  6  or  more  years 

25 

15 

Fig  4.4 
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Part  V:  Communications,  Access  and  Information  Requirements 

Information  Needs: 

The  majority  of  DTIC  users  who  responded  to  the  survey  indicated  that  they  were  able  to  find 
the  type  of  information  they  needed  in  DTIC's  collection.  (Reference  chart  on  page  43) 
Respondents  were  asked  to  provide  specific  comments  regarding  their  current  and  future 
informational  needs  from  which  a  number  of  topics  emerged.  Most  often  mentioned  was 
documentation  concerning  Homeland  Security.  Additional  information  needs  identified  below 
were  the  results  of  respondents'  most  frequently  mentioned  write-ins. 

■  Full-Text  Documentation 

■  Weapons  of  Mass  Destruction/Nuclear,  Biological,  Chemical 

■  Missile  Defense 

■  Aviation 


Communication  Preferences  and  Satisfaction: 


CS  survey  respondents  were  asked  to  identify  how  they  would  like  to  be  informed  about  DTIC 
products/services,  updates,  etc.  Eighty  percent  of  users  ranked  email  as  the  preferred  method. 
This  was  followed  by  eight  (8)  percent  of  respondents  requesting  information  via  listservs.  2004 
CS  respondent  preferences  for  email  contact  showed  an  eleven  point  increase  as  compared  to 
2002  data. 


a> 

O) 

ro 

■«-* 

c 

a> 

o 

i— 

0) 

CL 


FY  2002  CS 


FY  2003  CS 


FY  2004  CS 


■  Email 

□  Electronic  Mailing  List 

(listservs) _ 

■  Fax 

□  Web  Site 

■  Mail 


Survey  % 
69 
13 


0.2 

11 

6 

0.2 


Survey  % 


Survey  % 


N/A* 

N/A* 


80 

8 


N/A* 

N/A* 

N/A* 

N/A* 


0.2 

8 

3 

0.9 


I  □  Other 

Data  not  gathered 


Fig  5.1 
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Effectiveness  of  Corporate  Communications: 

Although  the  majority  of  respondents  (65  percent)  “agree”  or  “strongly  agree”  that  DTIC  does  an 
adequate  job  in  communicating  (transmitting/mailing)  information  about  new,  changed,  and/or 
enhanced  products  and  services  offerings,  etc.,  there  was  a  decline  in  the  ratings.  The  2004  CS 
survey  results  showed  each  of  the  DTIC  marketing  communication  efforts  declining  in  the 
ratings.  Communication  about  submihing  documents  and  training  opportunities  continue  to 
remain  the  two  lowest  scores. 


Comparative  Analysis  on  Effectiveness  of  Corporate  Communication 


FY  2002  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product/Service  Changes 

83% 

0.7% 

16% 

Product/Service  Changes 

82% 

2% 

16% 

Training  Opportunities 

71% 

3% 

27% 

Submitting  Documents 

59% 

4% 

37% 

Ordering  Documents 

77% 

3% 

20% 

System  Disruptions/Downtime 

76% 

3% 

22% 

Overall  Rating 

75% 

3% 

23% 

FY  2003  Users  Survey 

Strongly  Agree 

Strongly  Disagree  or 

No  Opinion 

or  Agree 

Disagree 

New  Product/Service  Changes 

81% 

5% 

14% 

Product/Service  Changes 

82% 

6% 

12% 

Submitting  Documents 

67% 

11% 

22% 

Training  Opportunities 

63% 

8% 

29% 

Ordering  Documents 

75% 

8% 

17% 

System  Disruptions/Downtime 

76% 

7% 

17% 

Overall  Rating 

74% 

8% 

18% 

FY  2004  Users  Survey 

Strongly  Agree 

Strongly  Disagree  or 

No  Opinion 

or  Agree 

Disagree 

New  Product/Service  Changes 

73% 

6% 

21% 

Product  /Service  Changes 

75% 

6% 

19% 

Submitting  Documents 

52% 

8% 

39% 

Training  Opportunities 

53% 

12% 

35% 

Ordering  Documents 

62% 

8% 

30% 

System  Disruptions/Downtime 

74% 

4% 

22% 

Overall  Rating 

65% 

7% 

28% 

Fig  5.2 
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Importance  of  Talking  with  a  "Live  Person" 

Fifty-four  percent  of  2004  CS  surveyed  respondents  preferred  to  speak  with  a  live  person  when 
contacting  DTIC.  Periodically  this  question  is  asked  in  order  to  stay  informed  of  users’  needs 
regarding  the  importance  of  talking  to  a  "live  person."  In  comparing  the  2002  and  2004  CS 
surveys  there  has  been  an  eight  point  drop  in  users  who  say  it  is  "Very  Important"  to  "Extremely 
Important"  to  speak  to  a  live  person.  Despite  the  decrease  in  the  percentage  of  respondents 
wishing  to  speak  to  a  live  person,  over  half  of  the  respondents  continued  to  express  the 
importance  of  “live  person"  telephone  contacts. 
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Comparative  Analysis  on  Importance  of 
Talking  to  a  Live  Person 


— 

1 

u- 

FY  2002  CS 
Survey  % 

FY  2003  CS 
Survey  % 

FY  2004  CS 
Survey  % 

■  Very  to  Extremely  Important 

62 

N/A* 

54 

□  Important 

23 

N/A* 

25 

□  Not  Very  Important  and/or 

Not  At  All  Important 

15 

N/A* 

21 

Data  not  gathered 


Fig  5.3 
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Part  VI:  2004  CSS  User  Comments 


The  comments  received  from  users  have  been  sorted  into  specific  categories  to  assist  in 
determining  problems  or  trends.  These  comments  were  selected  based  on  the  frequency  with 
which  an  issue  was  addressed,  either  positive  or  negative,  and  the  importance  of  the  issue  as 
determined  by  the  survey  analyst. 


j Customer  Service  Issues 


Comments 

Pro: 

■  "The  people  are  wonderful.  They  seem  to  really  care  about  me,  the  customer.  They  know 
their  job  and  strive  to  provide  the  best  possible  support.  They  are  friendly  and  considerate 
and  very  patient.  They  do  a  great  job  with  a  complex  system.” 

■  "DTIC  people  are  a  good  advertisement  for  what  government  should  be  -  helpful,  courteous, 
and  knowledgeable.” 

■  “DTIC  customer  support  is  the  best  of  any  government  agency  I  deal  with.” 

Con: 

■  "It  is  often  difficult  to  reach  a  person  in  the  customer  service  area.  We  have  had  a  lot  of 
problems  with  the  registration  process  which  the  service  reps  were  unable  to  understand.” 

■  "I  had  to  place  an  order  for  one  set  of  documents  four  different  times  before  I  actually 
received  it.  I  placed  it  on  the  Internet,  then  I  faxed  in  the  order  twice,  and  finally  I  called  it  in 
-  that’s  when  I  actually  received  it." 

■  “I  ordered  five  documents  weeks  ago  (requiring  use  of  Form  55)  and  have  no  response  from 
any  of  the  five.  This  seems  to  be  my  standard  experience.” 

■  "One  problem  with  DTIC  customer  service  is  that  I  often  do  not  know  who  to  contact 
concerning  the  problem  I  have,  it  is  sometimes  very  confusing.  I  also  find  that  email 
messages  get  no  response  so  I  would  rather  talk  to  someone  on  the  phone." 
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j Improvement  of  Products,  Services,  Customer  Care 


Comments 

Pro: 

■  "This  is  the  most  generous  use  of  resources  I  have  experienced  in  my  over  25  years  of 
government  service.  As  a  Librarian  I  am  please  with  the  service  and  the  products  you  offer. 
Thank  you." 

■  "Overall  I  am  quite  satisfied  with  DTIC  and  its  services.  Keep  up  the  good  work." 

Con: 

■  "Streamline  the  renewal  process." 

■  "Private  STINET  works  for  someone  who  is  ordering  documents  but  not  for  someone  who  is 
trying  to  search  for  documents  for  a  customer." 

■  “More  real  people  answering  their  telephones.” 

■  “The  posting  of  credit  card  transactions  needs  overhaul.” 

■  "Improve  the  search  capacities  for  STINET.  Searching  with  Boolean  logic,  past  search 
statements,  and  seeing  the  results  of  each  search  would  be  helpful." 

j  Online  Services  j 

Comments 

Pro: 

■  "DTIC  online  services  are  very  good  and  also  the  customer  support  is  very  helpful." 

■  "Overall  first  rate.  Only  occasionally  do  I  have  difficulty  in  locating  what  I  need.  Thanks." 

■  "Have  found  very  helpful  information  when  searching  online  for  documents,  MIL-SPECS, 
etc." 

■  “I  believe  this  organization  is  crucial  and  the  resources  available  will  be  a  tremendous  asset 
to  Homeland  Security.” 
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■  "I  think  DTIC  is  the  best  library  I  have  ever  utilized.  I  am  very  grateful  to  being  a  member 

59 


Con: 

■  "Ask  a  librarian/cataloger/indexer  to  organize  the  site.  It  is  not  intuitive  and  user  must  dig  to 
find  specific  collection.  Make  it  simple  and  put  a  search  box  similar  to  Google.  Improve 
Help  with  better  examples." 

■  "The  service  would  be  more  useful  if  bibliographic  information  were  kept  more  up  to  date." 

■  “I’ve  gotten  on  DTIC  one  time  looking  for  some  program  documentation.  I  can’t  remember 
finding  any  because  the  whole  system  was  confusing.  I  need  a  tutorial  on  DTIC. 

■  "Abstracts  provided  are  not  sufficient  to  detennine  whether  a  document  is  worth  getting.  I 
can’t  determine  the  usefulness  of  the  document.  This  is  especially  important  since  you  charge 
a  fee.  I  have  to  go  through  a  great  hassle  to  get  the  money  to  buy  a  product.” 


j  Quality  of  Products  and  Services 


Comments 

Pro: 

■  “Quality  is  excellent  and  very  professional.” 

■  “They’re  excellent  products  which  provide  a  broad  range  of  information  pertaining  to  current 
and  historic  ships  and  gun  systems,  etc.  to  which  I  need  access.” 

■  “Very  helpful  and  meet  needs  and  make  work  easier.” 

Con: 

■  "I  have  had  a  hard  time  searching  for  infonnation  in  the  Public  STINET  site." 

■  "What  I  am  really  looking  for  is  a  Google-like  interface  that  is  simple,  forgiving,  and  quick. 
Once  I’m  logged  on  to  the  Private  side  I  should  be  able  to  search  the  publicly  available 
documents  as  well  as  the  restricted  documents." 

■  "I  ordered  a  document  and  received  it  within  10  days.  The  invoice,  however,  took  about  30 
days  to  receive." 
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Part  VII:  Issues  for  Further  Study,  Conclusions  and  Recommendations 


1 .  The  following  suggestions  were  made  by  users  responding  to  the  survey: 


■  “I’ve  intended  to  sign  up  for  search  training  but  haven’t  had  time  available  to  go.  Would  like 
to  see  a  couple  of  scenario  based,  detailed  sample  searches  in  the  help  area  where  a 
researcher  goes  through  several  collections  to  find  specific  information.  I  see  this  going 
beyond  the  fine  tips  currently  available  in  Help.  Thanks  for  the  great  service  you  folks 
offer!” 

■  “My  only  complaint  is  that  the  file  size  of  the  PDFs  available  on  line  is  huge.  If  there  were 
any  way  to  reduce  the  file  size,  I  would  definitely  download  a  lot  more  of  my  documents  that 
way.  I  usually  just  end  up  ordering  the  hard  copies  because  it  takes  an  hour  or  so  to 
download  the  electronic  ones.” 

■  “Need  to  be  able  to  download  searches  as  Word  documents,  not  have  to  capture  the  HTML 
then  paste  it  into  Word  then  make  changes.” 

■  “We  would  very  much  appreciate  having  some  DTIC  training,  particularly  at  our  site,  if 
possible.  We  feel  that  we  are  not  using  the  resources  to  their  full  capabilities.” 

■  “I  would  like  to  see  more  flexibility  incorporated  into  Private  STINET.  I  would  very  much 
like  to  see  the  ability  to  mark  and  download  specific  records  or  at  the  very  least  be  able  to 
mark  and  print  records.  Private  STINET  works  for  someone  who  is  ordering  documents  but 
not  for  someone  who  is  trying  to  search  for  documents  for  a  customer.” 

■  “Get  rid  of  the  fee  for  government  users  who  need  this  to  do  their  job.  Barring  that  make  it 
possible  to  set  up  some  kind  of  organizational  membership  fee  that  covers  the  document  fees 
for  a  set  period  of  time  for  a  unit. . .” 

■  “Greater  speed  in  entering  new  documents.  We  required  a  copy  of  a  document  and  had  to  go 
to  the  originator  because  it  had  not  been  entered  even  though  it  had  been  over  six  months 
since  it  had  been  sent  to  DTIC.” 

■  “I  would  like  to  see  training  offered  as  well  as  some  sort  of  guide  to  searching  and  ordering.” 

■  “It  might  be  easier  to  use  this  great  resource  if  there  was  some  way  to  get  a  blanket  Form  55 
for  reports  of  the  same  nature  or  from  the  same  agency. . .  There  has  to  be  a  better  way  than 
one  form  and  approval  for  each  report. . .” 

■  “The  posting  of  credit  card  transactions  needs  overhaul.  It  is  impossible  to  reconcile  our 
credit  card  statements  when  each  item  purchased  is  posted  by  itself  with  no  way  to  tell  to 
which  specific  pub  the  charge  refers.  The  person  who  orders  most  of  our  DTIC  pubs  places 
six  orders  which  resulted  in  83  postings  to  his  credit  card. . .” 
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2.  The  average  number  of  non-deliverable  email  notifications  that  were  sent  out  in  FY  2004  was 
845  (returned  emails).  For  FY  2003  survey  average  was  824.  The  following  may  be  possible 
reasons  for  undeliverable  e-mails: 


•  E-mail  address  on  file  at  DTIC  not  valid 

•  Email  Program  Blockers 

•  User  no  longer  at  e-mail  address 

•  Server  problems  for  recipient  or  originator 

Recommendation:  Improvements  in  maintenance  of  contact  databases  and  registration 
information. 


3.  Low  response  rate  for  survey. 


Eighty-three  percent  of  2004  general  users  did  not  respond  to  the  survey.  The  call  effort  to 
increase  the  response  rate  wasn’t  as  intense  as  it  was  in  the  past  (due  to  staff  shortages).  The 
response  rate  of  17  percent  for  the  2004  CS  survey  effort  was  10  percentage  points  lower  than 
last  year's  rate. 

Recommendation:  PROCAP  staff  to  continue  their  combined  effort  to  reach  potential 
respondents  with  Web-based  surveys  for  the  first  contact.  There  should  be  two  reminders  then 
follow-ups  with  one-on-one  telephone  contacts  in  order  to  increase  response  rate. 


4.  Received  a  number  of  respondents  choosing  "Other"  to  describe  their  current  job  position. 


211  respondents  chose  the  option  "Other”  to  describe  their  current  job  positions.  For  example, 
some  of  the  write-ins  included:  Security  Officer,  Anti-Terrorism/Force  Protection  Program 
Manager,  Professor,  Student,  etc. 

Recommendation:  Minor  revision  of  the  job  position  description  will  be  looked  into. 


5.  There  were  7901  eligible  general  users  for  this  survey  effort.  Active  users  not  surveyed 
included  DTIC  employees,  users  identified  as  Top  200,  and  Air  Force  Materiel  Command 
(AFMC)  personnel.  (AFMC  personnel  were  not  surveyed  based  on  an  understanding  with  DTIC 
management  to  exclude  this  group). 


Recommendation:  AFMC  personnel  will  be  surveyed  commencing  with  the  FY  2005  Customer 
Satisfaction  Survey.  DTIC  and  other  users  will  benefit  from  the  input  submitted  by  these 
personnel. 


26 


6.  Users  were  surveyed  on  their  usage  of  DTIC's  two  major  online  services— Public  STINET  and 
Private  STINET.  The  results  of  the  2004  CS  surveyed  findings  show  Public  STINET  is  the  more 
widely  used  of  the  two  at  76  percent,  followed  closely  by  Private  STINET  at  72  percent. 


Recommendation:  All  surveyed  users  are  eligible  to  use  Private  STINET,  which  has  a  wider 
array  of  resources  than  Public  STINET.  Reasons  for  the  heavier  usage  of  Public  STINET  will  be 
examined  in  depth  if  similar  results  are  obtained  in  2005. 
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Appendix  A 

DTIC  Customer  Satisfaction  Questionnaire 


DTIC  2004  CUSTOMER  SATISFACTION  SURVEY  QUESTIONNAIRE 
OVERVIEW: 

Please  read  this  carefully  before  starting: 

This  data  collection  is  authorized  by:  RCS  DD-NII  (AR)  2051  &  OMB  5CFR  1320.5  (b) 
OMB  0704-0403,  Expires  08/31/04 

This  web-based  assessment  is  voluntary.  However,  your  input  is  essential  to  improving  our 
continuing  operations  and  helping  us  better  serve  your  information  needs.  The  average 
response  time  to  complete  this  survey  is  about  5  minutes.  Please  read  each  question 
carefully  before  responding. 

Thank  you  for  participating. 


SECTION  1:  USER  DEMOGRAPHICS 

1 .  Which  of  the  following  best  describes  your  current  organization?  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  DoD/Military/Civilian  (All  Services/Guard/Reserves/  Labs/Colleges/Schools) 

□  Non-DoD  Federal  Government  (All  Departments/Agencies/Labs) 

□  Industry/Corporate/Government  Contractor/Small  Business/  Labs 

□  Historical  Black  Colleges  and  Universities  and  Minority  Institutions  (HBCU/MI) 

□  College/University/Research  Center  (other  than  HBCU/MI) 

□  Other  (please  specify  in  the  space  below) 


Box  to  write  in  answer 


2.  In  general,  which  of  the  following  best  describes  your  current  job  position?  (Click  on  one) 
Multiple  Choice  (Single  Response) 

□  Contracting  Officer/Procurement  Specialist/COTR/Sponsor 

□  Engineer  (Aeronautical,  Civil,  Industrial,  Mechanical  or  other  Engineering  Fields) 

□  Librarian  (Head,  Technical,  Research  or  other  Library  Staff) 

□  Researcher/ Analyst/Program  Manager  (Non-Scientist) 

□  Resource  Management/Financial/Budget  Analyst 

□  Scientist  (Aerospace,  Biology,  Chemistry,  Physics  or  other  Scientific  Field) 

□  Technical  Infonnation  Specialist 

□  Other  (please  specify  in  the  space  below) 


Box  to  write  in  answer 
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3.  When  you  acquire  scientific  and  technical  information  from  DTIC,  who  uses  it?  (Click  on 
one) 

Multiple  Choice  (Single  Response) 

□  I  primarily  provide  information  to  others  (intermediary  user) 

□  I  use  the  information  myself  in  my  current  job  position  (end  user) 

4.  You  have  been  a  registered  DTIC  user  for:  (Click  on  one) 

Multiple  Choice  (Single  Response) 

□  Less  than  1  year 

□  1-2  years 

□  3-5  years 

□  6  or  more  years 


SECTION  2:  INFORMATION  REQUIREMENTS 

5.  Are  you  able  to  find  the  type(s)  of  information  that  you  need  in  the  DTIC  collection?  (Click  on 
one) 

Multiple  Choice  (Single  Response) 

□  Yes 

□  Not  Sure 

□  No 

6.  List  your  additional  current  and/or  future  information  needs: 

Box  for  Open  Ended-Write-in 


SECTION  3:  USERS  OF  DTIC  ONLINE  SERVICES 

Please  read  before  continuing  with  the  Survey 

Questions  7  through  10  are  to  be  completed  by  those  individuals  who  currently  use  the 
DTIC  online  services  listed  below.  If  you  do  not  use  either  of  the  online  services  listed,  skip 
to  question  11. 

7.  Which  of  the  following  DTIC  online  service(s)  do  you  currently  use?  (Click  on  as  many  as 
apply  to  you) 

Condensed  Question  List: 

□  Public  STINET 

□  Private  STINET 
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Multiple  Choice  (Single  Response) 

□  Yes 

□  Not  Sure 

□  No 


8.  If  you  use  Public  STINET,  please  rate  the  following  quality  factors: 

Condensed  Question  List: 

□  Accessibility  (Access  to  site): 

□  Ease  of  use  (Navigating  the  site): 

□  Content: 

□  Organization: 

□  Usability: 

□  Customer  Support: 

Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

□  Not  Sure 

9.  If  you  use  Private  STINET,  please  rate  the  following  quality  factors: 

Condensed  Question  List: 

□  Accessibility  (Access  to  site): 

□  Ease  of  use  (Navigating  the  site): 

□  Content: 

□  Organization: 

□  Usability: 

□  Customer  Support: 

Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

□  Not  Sure 

10.  Your  comments  about  the  quality  of  DTIC  online  services: 

Box  for  Open  Ended- Write-in 
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SECTION  4:  USERS  OF  OTHER  DTIC  PRODUCTS  AND  SERVICES 


Please  read  before  continuing  with  the  survey 

Questions  11  through  13  are  to  be  completed  by  those  individuals  who  currently  use  one  or 
more  of  the  DTIC  products  or  services  listed  below.  If  you  do  not  use  any  of  these  products 
or  services,  skip  to  question  16. 

1 1 .  What  DTIC  products  and  services  do  you  currently  use?  (Click  as  many  as  apply  to  you) 

Condensed  Question  List: 

□  ADD 

□  ECAB/ECAB-Docs 

□  CAB 

□  The  DTIC  Review 

□  Trail 

□  TR  Database  CD-ROM/DVD 

Response  Multiple  Choice  (Single  Response) 

□  Yes 

□  Not  Sure 

□  No 

12.  Please  rate  the  overall  quality  of  the  DTIC  products  and  services  that  you  currently  use: 
(Click  on  as  many  as  apply) 

Condensed  Question  List: 

□  ADD 

□  ECAB/ECAB-Docs 

□  CAB 

□  The  DTIC  Review 

□  Trail 

□  TR  Database  CD-ROM/DVD 

Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

□  Not  Sure 

13.  Your  comments  on  the  quality  of  the  above  DTIC  products  and  services: 

Box  for  Open  Ended-Write-in 
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SECTION  5:  CUSTOMER  SERVICE  EXPERIENCES 


14.  Given  our  goal  of  providing  maximum  access  while  protecting  national  security  interests, 
how  would  you  rate  your  overall  level  of  satisfaction  with  the  DTIC  registration  process?  (Click 
on  one) 

Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

15.  In  your  experience(s)  with  DTIC's  registration  process,  how  would  you  rate  your  satisfaction 
level  on  the  following  factors?  (Click  on  as  many  as  apply) 

Condensed  Question  List: 

□  Accessibility 

□  Ease  of  Use  (Navigating  the  Web-Based  registration  process)Spe 

□  Speed  of  Service 

□  Customer  Support 

Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

□  Not  Sure 

16.  Your  comments  about  DTIC’s  registration  process: 

Box  for  Open  Ended-Write-in 


17.  In  your  experience(s)  with  us  how  would  you  rate  DTIC,  as  a  whole,  on  the  following 
customer  service  quality  factors?  (Click  on  as  many  as  apply) 

Condensed  Question  List: 

□  Accessibility  (Easy  to  access  staff): 

□  Accuracy  (Provided  correct  infonnation): 

□  Courtesy: 

□  Helpfulness: 

□  Knowledge: 

□  Professionalism: 

□  Responsiveness  (Timely  response): 

□  Speed  of  Service  (How  quickly  performed): 
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Response  Multiple  Choice  (Single  Response) 

□  Excellent 

□  Very  Good 

□  Good 

□  Fair 

□  Poor 

18.  When  you  order  hard  copy  documents,  how  often  does  DTIC  meet  your  anticipated  delivery 
requirements?  (Click  on  one) 

Response  Multiple  Choice  (Single  Response) 

□  Every  time 

□  Most  of  the  time 

□  About  half  of  the  time 

□  Less  than  half  of  the  time 

□  Never 

□  Rarely,  if  ever,  order  hard  copy  documents 

19.  Your  comments  about  the  quality  of  DTIC  customer  care: 

Box  for  Open  Ended-Write-in 


SECTION  6  -  COMMUNICATIONS  AND  ACCESS 

20.  How  would  you  most  like  to  be  informed  about  DTIC  products,  services,  updates,  etc?  (Click 
on  one) 

Multiple  Choice  (Single  Response) 

□  E-mail 

□  Electronic  Mailing  List  (listservs) 

□  Fax 

□  Web  site 

□  Mail 

□  Other  (please  specify  in  the  space  below) 

Box  for  Open  Ended-Write-in 


21.  When  contacting  DTIC,  how  important  is  it  for  you  to  speak  with  a  "live  person"?  (Click  on 
one) 

Multiple  Choice  (Single  Response) 

□  Extremely  Important 

□  Very  Important 

□  Important 

□  Not  Very  Important 
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□  Not  at  All  Important 

22.  DTIC  does  a  good  job  infonning  you  about:  (Click  on  all  that  apply) 

Condensed  Question  List: 

□  New  Products/Services 

□  Product/Services  Changes 

□  Submitting  Documents 

□  Training  Opportunities 

□  Ordering  Documents 

□  System  Disruptions/Downtime 

Multiple  Choice  (Single  Response) 

□  Strongly  Agree 

□  Agree 

□  Disagree 

□  Strongly  Disagree 

□  No  Opinion 

SECTION  7  -  GENERAL 


23.  Do  you  have  a  requirement  for  classified  information? 

Multiple  Choice  (Single  Response) 

□  Yes 

□  No 

□  Not  Sure 

NOTE:  In  the  Fall  DTIC  plans  to  implement  Classified  STINET  on  the  SIPRNET.  For 
those  who  are  interested  in  using  DTIC's  classified  system  please  forward  your  inquiries  to: 
bcporder@dtic.mil. 

24.  How  important  are  DTIC  products/services  to  the  accomplishment  of  your  business 
objectives?  (Click  on  one) 

Multiple  Choice  (Single  Response) 

□  Extremely  Important 

□  Very  Important 

□  Important 

□  Not  Very  Important 

□  Not  at  All  Important 
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25.  Which  of  the  following  describes  DTIC's  impact  in  supporting  your  overall  mission?  (Click 
on  one) 


Multiple  Choice  (Single  Response) 

□  Very  High  Impact 

□  High  Impact 

□  Medium  Impact 

□  Low  Impact 

□  Very  Low  Impact 

26.  Would  you  recommend  DTIC  to  a  friend  or  colleague?  (Click  on  one) 

Multiple  Choice  (Single  Response) 

□  Yes 

□  Not  Sure 

□  No 

27.  Please  include  your  comments  and  suggestions  for  improving  DTIC  products,  services 

and/or  our  customer  care. _ 

Box  for  Open  Ended  -  Write-in 


NOTE: 

Thank  you  for  participating  in  this  important  survey. 


SUMMIT 


CLEAR 
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Appendix  B 
Quantitative  Results 


Reference  Fig  2.1  &  Fig  2.2 


2004  Customer  Satisfaction  Performance  Rating  (General  Users) 


OuetDoni 

Count 
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Mean  a*  a  percent  otponlble 
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O  20  40  60  80  100 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Easy  to  access  staff) 

1153 

72.00 

71.0% 

22.3% 

6.7% 

Accuracy  (Provided  correct  information) 

1152 

75.25 

76.5% 

19.2% 

4.3% 

Courtesy 

1132 

78.75 

79.4% 

17.8% 

2.8% 

Helpfulness 

1128 

76.75 

76.7% 

19.0% 

4.3% 

knowledge 

1118 

76.50 

76.9% 

18.9% 

4.2% 

Professionalism 

1122 

78.25 

79.6% 

17.2% 

3.2% 

Responsiveness  (Timely  response) 

1125 

74.50 

74.4% 

18.9% 

6.7% 

Speed  of  Service  (Hour  quickly  performed) 

1119 

73.75 

73.3% 

20.6% 

6.2% 

Overall  .Averages 

1131.1 

76.0% 

19.2% 

4.8% 

Reference  Fig  3.1,  3.2,  3.3 


Public  STINET  Quality  Performance  Rating 


Gueilloni 

Count 
(VSIId  N) 

Mean 
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Mean  ai  a  percent  of  ponlble 
icore 
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Favorable 
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Accessibility  (Access  to  site) 

860 

72.40 

67.4% 

22.9% 

9.7% 

Ease  of  use  (Navigating  the  site) 

855 

64.60 

50.5% 

36.4% 

13.1% 

Content 

856 

68.00 

59.7% 

29.2% 

11.1% 

Organization 

854 

64.60 

51.6% 

36.2% 

12.2% 

Usability  (Information  useful) 

853 

66.20 

55.7% 

31.5% 

12.8% 

Customer  Support 
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57.20 

53.0% 

17.2% 

29.8% 

Overall  Averages 
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56.3% 

28.9% 

14.8% 
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Reference  Fig  3.2,  3.3 


Private  STINET  Quality  Performance  Rating 


Cueitoni 

Oount 
(Ittiid  N) 

Mean 

(norm.) 

Mean  3i  a  percent  of  poiilble 
icore 

□  a]  «o  m  a]  ice 

Favorable 

Neutral 

Unfavorable 

Accessibility  (Access  to  site) 

867 

70.40 

66.4% 

19.7% 

13.8% 
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858 

69.00 
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23.5% 
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850 
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Customer  Support 

854 
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54.0% 
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Overall  Averages 
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58.4% 

24.7% 

16.9% 

Reference  Fig  3.1 


Which  of  the  following  DTIC  online  service(s)  do  you  currently  use?  (Click  on  as 
many  as  apply  to  you) 


Public  STINET 


Private  STINET 
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Reference  Fig  3.4 


What  DTIC  products  and  services  do  you  currently  use?  (Click  as  many  as  apply 
to  you) 


ADD 


ECAB/ECAB-Docs 


Response 

Count 
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0  20  4 

0  6 

0  8 

0  100 

Yes 
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No 
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Not  Sure 
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40.0% 

CAB 
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The  DTIC  Review 


Response 

Count 

Percent 

0  2 

Yes 
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29.5% 

No 
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37.5% 

Not  Sure 
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33.0% 

■ 

60 


80  100 


TRAIL 


TR  Database  CD-ROM/DVD 
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Reference  Fig  3.5 


Overall  quality  of  the  DTIC  products/services 


Com 
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0.4% 
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03% 
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Reference  Fig  3.6 


Given  our  goal  of  providing  maximum  access  while  protecting  national  security 
interests,  how  would  you  rate  your  overall  level  of  satisfaction  with  the  DTIC 
registration  process?  (Click  on  one) 


Response 

Count 

Percent 

0  20  4 

0  6 

0  8 

0  100 

Excellent 

289 

26.1% 

Very  Good 

467 

42.1% 

Good 

263 

23.7% 

1 

Fair 

Poor 

60 

30 

5.4% 

2.7% 

r 
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Reference  4.1 


Which  of  the  following  best  describes  your  current  organization?  (Click  on  one) 
(Scale  1) 


Response 

Count 

Percent 

0  20  40  60  8 

D  100 

DoD/Military/Civilian  (All 

845 

64.5% 

Non-DoD  Federal  Government  (All 

50 

3.8% 

Industry/Corporate/Government 

350 

26.7% 

Historically  Black  Colleges  and 

17 

1.3% 

College/University/Research  Center 

28 

2.1% 

Other  (please  specify  in  the  space 

20 

1.5% 

1 

Reference  Fig  4.2 


In  general,  which  of  the  following  best  describes  your  current  job  position?  (Click 
on  one)  (Scale  1) 


Response 

Count 

Percent 

Contracting  Officer/Procurement 

40 

3.1% 

Engineer  (Aeronautical,  Civil, 

274 

21 .4% 

Librarian  (Head,  Technical,  Research 

231 

18.1% 

Researcher/Analyst/Program  Manager 

249 

19.5% 

Resource 

31 

2.4% 

Scientist  (Aerospace,  Biology, 

119 

9.3% 

Technical  Information  Specialist 

123 

9.6% 

Other  (please  specify  in  the  space 

211 

16.5% 

40 


60 


80  100 
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Reference  Fig  4.3 


When  you  acquire  scientific  and  technical  information  from  DTIC,  who  uses  it? 
(Click  on  one) 


Response 

Count 

Percent 

0  20  40  6 

0  8 

0  100 

1  obtain  the  information  for  others 

1  obtain  the  information  for  myself 

530 

760 

41.1% 

58.9% 

Reference  Fig  4.4 


You  have  been  a  registered  DTIC  user  for:  (Click  on  one) 


42 


Are  you  able  to  find  the  type(s)  of  information  that  you  need  in  the  DTIC 
collection?  (Click  on  one) 


Response 

Count 

Percent 

Yes 

976 

75.1% 

No 

96 

7.4% 

Not  Sure 

227 

17.5% 

80  100 


3 


Reference  Fig  5.1 


How  would  you  most  like  to  be  informed  about  DTIC  products,  services,  updates, 
etc?  (Click  on  one)  (Scale  1) 


Response 

Count 

Percent 

0  20  40  60  80  100 

| | | | 

Email 

1000 

80.2% 

Electronic  Mailing  List  (listservs) 

102 

8.2% 

■ 

Fax 

3 

0.2% 

Web  site 

99 

7.9% 

■ 

Mail 

32 

2.6% 

1 

Other  (please  specify  in  the  space 

11 

0.9% 

43 


Reference  Fig  5.2 


DTIC  Does  a  Good  Job  informing  You  About: 


Cue  done 

Count 
•valid  N> 

Mean 
•norm  • 

Mean  a  c  a  peroentotpo  celble 
ooore 

b  30  4b  Ui  Mi  u«i 

ctongiy 

Agree 

Agree 

C<  cagree 

a  :  lull 

C<  cagree 

No  Cf»lnlon 

New  P  iodic*  Qe  rates 

1202 

6350 

22.7% 

50.7% 

4.7% 

13% 

203% 

P  rodictCe  lutes  C  laiges 

1200 

6550 

252% 

49.1% 

4.7% 

1.1% 

190% 

Cilun  rang  Docimeit 

1172 

4625 

126% 

39.7% 

72% 

1.1% 

39.4% 

Trail  lig  Opfaorti  ■  roes 

1180 

48.75 

142% 

392% 

9.7% 

1.9% 

35.1% 

Oiderlig  Decline  it 

1177 

5450 

172% 

45.1% 

63% 

13% 

302% 

C^tin  Dlsuptlois/Dcwfiltne 

1188 

6 525 

31:9% 

41.9% 

3.7% 

06% 

21.9% 

Oue  rail  A#e  rages 

1135.5 

5729 

208% 

443% 

60% 

12% 

27.7% 

Reference  Fig  5. .3 


When  contacting  DTIC,  how  important  is  it  for  you  to  speak  with  a  "live  person"? 
(Click  on  one) 


Response 

Count 

Percent 

0  20  4 

0  6i 

0  8 

0  100 

Extremely  Important 

327 

26.2% 

Very  Important 

345 

27.6% 

Important 

Not  Very  Important 

Not  at  All  Important 

316 

220 

42 

25.3% 

17.6% 

3.4% 

44 


Reference  Fig  2.5 


When  you  order  hard  copy  documents,  how  often  does  DTIC  meet  your 
anticipated  delivery  requirements?  (Click  on  one) 


Response 

Count 

Percent 

0  2 

0  4 

0  6 

0  8 

0  100 

Every  time 

201 

17.8% 

Most  of  the  time 

280 

24.8% 

About  half  of  the  time 

25 

2.2% 

1 

Less  than  half  of  the  time 

16 

1.4% 

Never 

46 

4.1% 

1 

Rarely,  if  ever,  order  hard  copy 

562 

49.7% 

Reference  Fig  2.4 


How  important  are  DTIC  products/services  to  the  accomplishment  of  your 
business  objectives?  (Click  on  one) 


Response 

Count 

Percent 

0  20  4 

0  6 

0  8 

0  100 

Extremely  Important 

259 

20.9% 

Very  Important 

345 

27.8% 

Important 

485 

39.0% 

Not  Very  Important 

Not  at  All  Important 

130 

23 

10.5% 

1.9% 
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Reference  Fig  2.5 


Which  of  the  following  describes  DTIC's  impact  in  supporting  your  overall 
mission?  (Click  on  one) 


Reference  Fig  2.6 


Would  you  recommend  DTIC  to  a  friend  or  colleague?  (Click  on  one) 


Response 

Count 

Percent 

0  20  40  60  80  100 

Yes 

1106 

88.6% 

No 

33 

2.6% 

Not  Sure 

109 

8.7% 

Registration 


(kMm 

C  MV 

iMfeirt 

nwv 

NNnMtfMKM  tfpnuttuo* 

•Me,  ijecrt 

f  tm 

nc/ 

Access  blllV 

1108 

73J30 

233% 

403% 

240% 

50% 

2.7% 

39% 

Ease  or  Use  rNarlgatlig  tie 

Web-Based  reg  (straltoi  process) 

1 1Q2 

69*0 

190% 

39.1% 

25.1% 

72% 

33% 

5.4% 

Cpeedoroeivfce 

1099 

7220 

249% 

37  j6% 

23.9% 

5j6% 

2.7% 

53% 

CistmerSipport 

1095 

69.40 

280% 

35.1% 

17.7% 

40% 

20% 

11.4% 

Oue  rail  e*  rages 

1 1013 

7105 

240% 

380% 

22.9% 

50% 

2.9% 

63% 
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Do  you  have  a  requirement  for  classified  information? 


Response 

Count 

Percent 

Yes 

600 

47.8% 

No 

459 

36.6% 

Not  Sure 

195 

15.6% 

60 


80  100 


Overall  Registration  Satisfaction  Level 


Gueitloni 

Oount 
(Vftlld  N) 

Mean 

(norm.) 

Mean  ai  a  percent  of  poiilble 
icore 

□  2]  40  33  33  ICE 

Favorable 

Neutral 

Unfavorable 

Given  our  goal  of  providing  maximum 
access  while  protecting  national  security 
interests,  how  would  you  rate  your  overall 
level  of  satisfaction  with  the  DTIC 
registration  process?  (Click  on  one) 

1109 

70.75 

68.2% 

23.7% 

8.1% 

Overall  Averages 

1109.0 

70.75 

68.2% 

23.7% 

8.1% 
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